Processing an infrared (ir) image based on swipe gestures

ABSTRACT

A computer-implemented method and a computer system for processing an infrared (IR) image based on a swipe gesture are provided. For example, the method may comprise: displaying an IR image within one or more graphical objects displayed on a touch screen; receiving a user indication of a swipe gesture via said touch screen, wherein receiving a user indication further comprises generating first data representing a first swipe gesture starting location and second data representing a first swipe gesture direction; processing the one or more graphical objects, wherein the processing comprises modifying various parameters or attributes associated with the IR image and/or image processing the IR image based on the first and the second data; and displaying the one or more processed graphical objects including the IR image processed according to the modified parameters or attributes.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is continuation of U.S. patent application Ser. No.14/575,110 filed Dec. 18, 2014 and entitled “PROCESSING AN INFRARED (IR)IMAGE BASED ON SWIPE GESTURES” which is hereby incorporated by referencein its entirety.

U.S. patent application Ser. No. 14/575,110 claims the benefit of U.S.Provisional Patent Application No. 61/991,394 filed May 9, 2014 andentitled “PROCESSING AN INFRARED (IR) IMAGE BASED ON SWIPE GESTURES”which is hereby incorporated by reference in its entirety.

U.S. patent application Ser. No. 14/575,110 claims the benefit of U.S.Provisional Patent Application No. 61/917,796 filed Dec. 18, 2013 andentitled “PROCESSING AN INFRARED (IR) IMAGE BASED ON SWIPE GESTURES”which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

Generally, embodiments of the invention relate to the technical field ofimage processing and visualization of infrared (IR) images, and inparticular, to image processing and visualization of infrared (IR)images for image analysis using swipe gestures.

BACKGROUND

Thermal, or infrared (IR), images of scenes are often useful formonitoring, inspection and/or maintenance purposes.

Typically, a thermal imaging device, e.g. in the form of a thermographyarrangement or an infrared IR camera, is provided to capture infrared(IR) images as IR image data values, representing infrared radiationemitted from an observed real world scene. The captured IR image canafter capturing be displayed or visualized and analyzed on a display ina computer system such as a thermography arrangement, infrared IRcamera, smartphone, tablet computer, laptop computer, desktop computeror a wrist-worn computer configured with a touch screen with a touchbased user input functionality.

As infrared radiation is not visible to the human eye there are nonatural relation between the captured infrared (IR) image's data valuesof each pixel in an IR image and greyscale or colors displayed on adisplay. Therefore an information visualization image processingreferred to as false color or pseudo color is used to map capturedinfrared (IR) image data values of each pixel in an IR image to apalette used to present the corresponding pixel displayed on a display,e.g. using grey-scale or colors.

Further, as IR images by nature are generally low contrast and noisy,the captured IR image may be subjected to further imaging processing into improve the interpretability of the image before displaying it to auser, such as filtering or combination/fusion with visual light imagescaptured of the same observed real world scene.

Applying an appropriate imaging processing to an IR image for display toa user is a cumbersome process as the requirements may varysignificantly with the observed real world scene, objects of interest inthe observed real world scene, the type of application or use case anduser preference.

There is therefore a need to reduce complexity and time for a user tofind an appropriate imaging processing to apply to an IR image fordisplay to a user, in particular in a computer system configured with atouch screen with a touch based user input functionality. Thereby theinterpretability of an IR image is improved and particular aspects ordetails of the image may be analyzed.

SUMMARY

Various techniques are provided for a computer system and acomputer-implemented method to process an infrared (IR) image based on aswipe gesture. For example, the method includes displaying an IR imageon a touch screen, wherein the IR image is displayed within a firstgraphical object of one or more graphical objects displayed within saidtouch screen; receiving, by a processor communicatively coupled to saidtouch screen, a user indication of a swipe gesture via said touchscreen, wherein said receiving the user indication comprises generatingfirst data representing a first swipe gesture starting location andsecond data representing a first swipe gesture direction; processing theone or more graphical objects based on the first and the second data,wherein the processing comprises at least one of: (i) splitting said IRimage within the first graphical object into a first and second parts toprocess the first or the second part, (ii) modifying a size of a secondgraphical object to a predefined expanded size or a predefined minimizedsize, (iii) modifying a temperature range for processing the IR image,or (iv) modifying one or more parameter ranges for processing the IRimage; and displaying said processed one or more graphical objectswithin said touch screen.

In another embodiment, a method may include: displaying an IR imagewithin a first graphical object displayed in a first set ofnon-overlapping graphical objects within a touch screen by applying afirst palette, receiving a user indication of a swipe gesture via saidtouch screen, wherein receiving a user indication further comprisesgenerating first data representing a first swipe gesture startinglocation and second data representing a first swipe gesture direction,splitting said IR image into two parts based on said first data, whereinthe two parts comprise a first part and a second part, selecting saidfirst part or said second part based on said second data, imageprocessing said selected first part or second part based on said seconddata, displaying said image processed first part or displaying saidimage processed second part overlaid onto said displayed IR image withinsaid touch screen.

In another embodiment, a method may include: displaying a graphicalobject within a touch screen, wherein said graphical object represents atemperature range for processing an IR image, and wherein the graphicalobject comprises one or more portions each associated with a temperaturevalue from the temperature range; receiving, by a processorcommunicatively coupled to said touch screen, a user indication of aswipe gesture via said touch screen, wherein said receiving the userindication comprises generating first data representing a swipe gesturestarting location and second data representing a swipe gesturedirection, and wherein said swipe gesture starting location correspondsto one of the portions of the graphical object; modifying said firsttemperature range based on said first data and said second data, whereinsaid modifying said temperature range comprises modifying one or more ofa minimum temperature value, a middle temperature value, and a maximumtemperature value of said temperature range; processing said IR imagebased on said modified temperature range; and displaying said processedIR image within said touch screen.

In another embodiment, a method may include: displaying the IR image ona touch screen, wherein the IR image is displayed within a firstgraphical object displayed within the touch screen; receiving, by aprocessor communicatively coupled to said touch screen, a userindication of a swipe gesture via said touch screen, wherein saidreceiving the user indication comprises generating first datarepresenting a swipe gesture starting location and second datarepresenting a swipe gesture direction, and wherein said second dataindicates that the user indication corresponds to a stationary swipe;determining, based at least in part on the first data representing theswipe gesture starting location, a spot marker location for selecting apixel or group of pixels of the IR image displayed within the firstgraphical object; displaying a second graphical object representing aspot marker by overlaying the second graphical object onto the IR imageat the spot marker location; and processing said IR image based on thespot marker location.

In another embodiment, a method may include: displaying a firstgraphical object comprising the IR image within a touch screen;displaying a second graphical object within said touch screen, whereinsaid second graphical object is overlaid onto the first graphicalobject; receiving, by a processor communicatively coupled to said touchscreen, a user indication of a swipe gesture via said touch screen,wherein said receiving the user indication comprises generating firstdata representing a swipe gesture starting location and second datarepresenting a swipe gesture direction, and wherein the swipe gesturestarting location corresponds to a coordinate associated with saidsecond graphical object; and modifying said second graphical objectbased on said first data and said second data, wherein modifying saidsecond graphical object comprises modifying the size of the secondgraphical object to a predefined expanded size or a predefined minimizedsize.

In another embodiment, a method may include: displaying the IR image ona touch screen based on a first parameter range and a second parameterrange, wherein the IR image is displayed within a first graphical objectdisplayed within the touch screen; displaying a second graphical objectand a third graphical object within said touch screen, wherein saidsecond graphical object represents the first parameter range andcomprises one or more portions each associated with a parameter valuefrom the first parameter range, and wherein said third graphical objectrepresents the second parameter range and comprises one or more portionseach associated with a parameter value from the second parameter range;receiving, by a processor communicatively coupled to said touch screen,a user indication of a swipe gesture via said touch screen, wherein saidreceiving the user indication comprises generating first datarepresenting a swipe gesture starting location and second datarepresenting a swipe gesture direction, and wherein the swipe gesturestarting location corresponds to one of the portions of the second orthe third graphical object; modifying said first or said secondparameter range based on said first data and said second data, whereinsaid modifying said first or said second parameter range comprisesmodifying one or more of a minimum parameter value, a middle parametervalue, and a parameter temperature value of said first or said secondparameter range; processing said IR image based on said modified firstor said modified second parameter range; and displaying said processedIR image within said touch screen.

In another embodiment, a non-transitory computer readable medium maystore computer readable code which, when executed by a processor, causesthe processor to perform the various methods of processing an infrared(IR) image based on a swipe gesture discussed above.

In another embodiment, a computer system may include: a memory; a touchscreen; and a processor communicatively coupled to the memory and thetouch screen, wherein said processor is configured to: display an IRimage on said touch screen, wherein the IR image is displayed within afirst graphical object of one or more graphical objects displayed withinsaid touch screen, receive a user indication of a swipe gesture via saidtouch screen, wherein said receiving the user indication comprisesgenerating first data representing a first swipe gesture startinglocation and second data representing a first swipe gesture direction,process the one or more graphical objects based on the first and thesecond data by performing one or more operations comprising at least oneof: (i) splitting said IR image within the first graphical object into afirst and second parts to process the first or the second part, (ii)modifying a size of a second graphical object to a predefined expandedsize or a predefined minimized size, (iii) modifying a temperature rangefor processing the IR image, or (iv) modifying one or more parameterranges for processing the IR image; and display said processed one ormore graphical objects within said touch screen.

The scope of the invention is defined by the claims, which areincorporated into this section by reference. A more completeunderstanding of embodiments of the invention will be afforded to thoseskilled in the art, as well as a realization of additional advantagesthereof, by a consideration of the following detailed description of oneor more embodiments. Reference will be made to the appended sheets ofdrawings that will first be described briefly.

BRIEF DESCRIPTION OF DRAWINGS

Embodiments of the invention will now be described in more detail withreference to the appended drawings, wherein:

FIG. 1 shows a schematic view of a computer system in accordance withone or more embodiments of the present disclosure.

FIG. 2 shows a schematic view of the computer system in accordance withone or more alternative embodiments of the present disclosure.

FIG. 3a illustrates how an IR image is processed based on a swipegesture, according to an embodiment of a computer-implemented method ofthe present disclosure.

FIG. 3b illustrates how an IR image is processed based on a swipegesture, according to another embodiment of a computer-implementedmethod of the present disclosure.

FIG. 4a shows an example image being displayed within a touch screenaccording to an embodiment of a computer-implemented method of thepresent disclosure.

FIG. 4b shows another example image illustrating how an IR image issplit within the touch screen according to an embodiment of acomputer-implemented method of the present disclosure.

FIG. 4c shows another example image being displayed within the touchscreen according to an embodiment of a computer-implemented method ofthe present disclosure.

FIG. 5a shows an example image being displayed within a touch screenaccording to an embodiment of a computer-implemented method of thepresent disclosure.

FIG. 5b shows another example image illustrating how an IR image issplit within the touch screen according to an embodiment of acomputer-implemented method of the present disclosure.

FIG. 5c shows another example image being displayed within the touchscreen according to an embodiment of a computer-implemented method ofthe present disclosure.

FIG. 6 illustrates how an image processing operation may be selectedbased on a determination of a direction of a swipe gesture, according toan embodiment of a computer-implemented method of the presentdisclosure.

FIG. 7a shows an example image illustrating how an IR image is processedbased on a swipe gesture including a stationary swipe or tap gesture,according to an embodiment of a computer-implemented method of thepresent disclosure.

FIG. 7b shows an example image that is resized based a swipe gesture,according to an embodiment of a computer-implemented method of thepresent disclosure.

FIG. 8 shows a flowchart of a computer-implemented method to process anIR image based on a swipe gesture, in accordance with an embodiment ofthe present disclosure.

FIG. 9 shows various example parameters for processing or presenting IRimages in a thermal imaging device display, in accordance with anembodiment of the disclosure.

FIGS. 10a and 10b show example images of an IR image and an associatedtemperature range that may be displayed within a touch screen, inaccordance with an embodiment of the disclosure.

FIGS. 11a, 11b, 12a, 12b, 13a, and 13b show touch screen imagesillustrating various examples of how a swipe gesture from a user may bereceived to modify a temperature range for processing or presenting anIR image, in accordance with one or more embodiments of the disclosure.

FIG. 14 shows a flowchart of a computer-implemented method to process anIR image based on a swipe gesture, in accordance with an embodiment ofthe disclosure.

FIGS. 15a, 15b, 16a, 16b, 17a, and 17b show touch screen imagesillustrating various examples of how a user indication of a swipegesture may be received to place or otherwise control a spot marker forprocessing an IR image, in accordance with one or more embodiments ofthe disclosure.

FIG. 18 shows an example touch screen image displaying an IR image and aspot marker, in accordance with one or more embodiments of thedisclosure.

FIG. 19 shows a flowchart of a computer-implemented method to process anIR image based on a swipe gesture, in accordance with an embodiment ofthe disclosure.

FIGS. 20a and 20b show touch screen images illustrating an example ofhow a size of a graphical object may be adjusted based on a swipegesture, in accordance with an embodiment of the disclosure.

FIG. 21 show a flowchart of a computer-implemented method to processgraphical objects displayed on a touch screen based on a swipe gesture,in accordance with an embodiment of the disclosure.

FIGS. 22a, 22b, 23a, and 23b show touch screen images illustratingvarious example of how a swipe gesture from a user may be received toadjust one of a plurality of parameters that control or affectprocessing and/or presentation of an IR image, in accordance with one ormore embodiments of the disclosure.

FIG. 24 shows a flowchart of a computer-implemented method to process anIR image based on a swipe gesture, in accordance with an embodiment ofthe disclosure.

Embodiments of the invention and their advantages are best understood byreferring to the detailed description that follows. It should beappreciated that like reference numerals are used to identify likeelements illustrated in one or more of the figures.

DETAILED DESCRIPTION Introduction

The invention relates to image analysis of IR images based on imageprocessing and visualization of infrared (IR) images for image analysisusing swipe gestures in a computer system. After capturing an IR imageby imaging systems integrated in or coupled to a computer system, a usertypically analyzes the image, e.g. to detect electric faults, heatemission or gas emission. The analysis may be performed in athermography arrangement, infrared IR camera, smartphone, tabletcomputer, laptop computer, desktop computer or a wrist-worn computerconfigured with a touch screen with a touch based user inputfunctionality.

As IR images typically are noisy and low contrast, the user may have toapply various image processing operations to be identifying areas ofinterest, such as overheated electrical components, faulty wallinsulation or gas leaks.

By applying a different palette or by applying a different image mode,such areas of interest may be visualized more clearly to the user, andthus improving the analysis of the IR image.

To apply image processing, e.g. apply a different palette or to apply adifferent image mode, in conventional systems is a cumbersome and timeconsuming process, that may include substantial menu navigation andparameter updates. Another problem is the delay between when a userfirst observes an IR image where a first palette has been applied andwhen the user observes the updated IR image where a second palette hasbeen applied. Yet another problem is that no comparison between thevisualizations is available.

The invention disclosed herein addresses the problems listed above andother problems related to analysis of IR images based on imageprocessing and visualization of infrared (IR) images for image analysis.

Applying a Palette to an Image or IR Image

As thermal images by nature are generally low contrast and noisy, thecaptured IR image may be subjected to various imaging processing in toimprove the interpretability of the image before displaying it to auser. Examples of such image processing is correction with IRtemperature calibration data parameters, low pass filtering,registration of multiple successive IR images and averaging to obtain aaveraged IR image or any other IR image processing operation known to aperson skilled in the art.

As infrared radiation is not visible to the human eye there are nonatural relation between the captured infrared (IR) image's data valuesof each pixel in an IR image and greyscale or colors displayed on adisplay. Therefore an information visualization process referred to asfalse color or pseudo color is used to map captured infrared (IR) imagedata values of each pixel in an IR image to a palette used to presentthe corresponding pixel displayed on a display, e.g. using grey-scale orcolors.

A palette is typically a finite set of color or grey-scalerepresentations selected from a color model for the display of images orvisual representations of IR images, i.e. a pre-defined paletterepresents a finite set of grayscale or color values of a color modeldisplayable on a display thereby making it visible to the human eye.

Mapping of captured infrared (IR) image data values of each pixel in anIR image to a palette also referred to as applying a palette, used topresent the corresponding pixel of a visual representation of said IRimage displayed on a display is typically performed by applying apre-determined relation, wherein said pre-determined relation describesa mapping from intervals of infrared image data values to saidpre-defined palette, e.g. a palette index value with an associated coloror grey-scale representation selected from a color model.

The captured IR image is typically displayed to an intended user basedon the captured IR image comprising infrared (IR) image data values, IRtemperature calibration data parameters, a predefined paletterepresenting a finite set of grayscale or color values of a color modeldisplayable on a display and a pre-determined relation describing amapping from infrared image data values to said pre-defined palette.

The same method of applying a palette to an image may be used for anytype of image, as would be understood by a person skilled in the art.

Image Mode

An IR image may in some embodiments comprise additional informationrelated to the observed real world scene. Example of such relatedinformation may be a visible light (VL) image (also referred to as avisual light image) depicting the same real world scene, a blueprint ordrawing of the observed real world scene, a distance map describingestimated distances to the observed real world scene, GPS coordinates,ambient temperature, moisture levels or other information related to theobserved real world scene. Such an IR image containing additionalinformation may also be referred to as a multilayer image.

The additional information may be either displayed or visualized aloneor be combined through image processing with the IR image as a combinedimage and displayed or visualized. Such display of additionalinformation alone or as a combined image is referred herein as an imagemode of the IR image.

Such image modes of the IR image may image processed beforehand andstored as an IR image or image processed instantaneously when applyingan image mode to an IR image.

One example of an image mode is VL only, wherein only additionalinformation from a VL image is displayed as an IR image within a firstgraphical object displayed in a first set of non-overlapping graphicalobjects within a screen (e.g., a touch screen), as would be understoodby a person skilled in the art.

Yet another example of an image mode is fusion, wherein IR image datavalues additional information from a VL image is aligned, superimposedand combined as a combined image by fusing or blending, e.g. by alphablending or other fusion techniques, as would be understood by a personskilled in the art.

Yet another example of an image mode is contrast enhancing fusion,wherein IR image data values additional information from a VL image isaligned, high pass filtering the VL image to extract pixel datarepresenting contours and/or edges in the visual image, combiningluminance information of the extracted pixel data with luminanceinformation of corresponding pixels in the IR image to augment the IRimage with the contours and/or edges from the visual image, as would beunderstood by a person skilled in the art.

Yet another example of an image mode is distance map, wherein IR imagedata values additional information from a VL image is used to calculatethe distance z from the thermal imaging device to the observed realworld scene for each pixel and used to form a distance map. In oneexample, the distance z is dependent on a predetermined displacementbetween the visible light imaging system and the IR imaging system.

Yet another example of an image mode is blueprint, wherein additionalinformation representing a blueprint or drawing of the observed realworld scene is displayed as an IR image within a first graphical objectdisplayed in a first set of non-overlapping graphical objects withinsaid touch screen, as would be understood by a person skilled in theart.

Color Range

Applying a palette might in some embodiments comprise applying a firstor a second palette only to IR image data values of an IR image that arewithin a particular range referred to as color range. The color rangemay be adapted by adapting a low threshold and/or a high threshold,wherein the low threshold and the high threshold define the color range.The color range may be adapted by simultaneously increasing ordecreasing the low threshold and the high threshold with the same value,thereby transposing the color range.

The color range may be adapted by increasing or decreasing the lowthreshold or the high threshold individually or simultaneously, therebyincreasing or reducing the color range.

Detecting a User Indication as a Swipe Gesture

Generally, computer systems that include or comprise a touch screen maydetect user indications made on the touch screen and generate dataindicative of user input. A touch screen enables a user to interact byuser indications with what is displayed, e.g. graphical objectsdisplayed on the touch screen using various types of gestures, such asswipe, pinch close, pinch open, tap, double tap, long press, flick,rotate. A touch screen may be configured to operate on a selection oftechnology, such as resistive, Surface acoustic wave (SAW), capacitive,surface capacitive, projected capacitive, mutual capacitance,self-capacitance or any other touch screen technology, as understood bya skilled person.

An image, e.g. an IR image, may be displayed in a graphical object, suchas a frame or image window, to a user of the computer system and agesture might be detected and converted to data indicative of thedetected gesture, as would be understood by a person skilled in the art.

A swipe gesture may be detected via said touch screen by receiving auser indication and generating a first data representing a first swipegesture starting location and second data representing a first swipegesture direction, wherein the first swipe gesture starting locationcorresponds to a first swipe gesture starting location coordinate withinsaid first graphical object displayed within said touch screen.Alternatively, the first swipe gesture starting location may correspondto a first swipe gesture starting location coordinate within said touchscreen. In one example, this might involve generating data correspondingto a swipe gesture starting location on the right hand part of the touchscreen and a leftward swipe gesture direction.

System Embodiments

As described above a user might be operating a computer system toanalyze a visualized IR image for analysis. To perform a successfulanalysis the user might apply various image processing operations.

FIG. 1 shows a schematic view of one or more embodiments of a computersystem, e.g. in the form of a thermography arrangement, infrared IRcamera, smartphone, tablet computer, laptop computer, desktop computeror a wrist-worn computer. Said computer system is configured forprocessing an infrared (IR) image based on a swipe gesture. The computersystem further comprises a processor/processing unit 112 provided withspecifically designed programming or program code portions adapted tocontrol the processing unit 112 to perform the steps and functions ofembodiments of the method described herein. The computer system furthercomprises at least one memory 115 configured to store data values orparameters received from the processor 112 or to retrieve and send datavalues or parameters to the processor 112. In one or more embodimentsthe computer system further comprises a display 117 configured toreceive a signal from a processor 112 and to display the received signalas a displayed image, e.g. to a user of the computer system. In one ormore embodiments the computer system 110 further comprises an inputdevice 118 configured to receive input or indications from a user, e.g.a user to indicate a swipe gesture in an image. In one exemplaryembodiment the display 117 is integrated with a user input device 118and implemented as a touch screen configured to receive a signal fromthe processor 112 and to display the received signal as a displayedimage, e.g. to a user of the computer system. The touch screen (e.g.,display 117) is further configured to receive indications from a userand to generate data indicative of user input, thereby enabling the userto interact by user indications with what is displayed, e.g. graphicalobjects displayed on the touch screen using various types of gestures.The touch screen is further configured to send the generated data as asignal to said processor 112. In one or more embodiments computer systemfurther comprises a communications interface 116 configured to send orreceive data values or parameters to/from a processor 112 to/fromexternal units via the communications interface 116.

In one or more embodiments, a computer system having a processoroperatively coupled to a memory and a touch screen for processing aninfrared (IR) image based on a swipe gesture, the system comprising:

-   -   a memory;    -   a touch screen; and    -   a processor, wherein said processor is configured to:    -   display an IR image within a first graphical object displayed in        a first set of non-overlapping graphical objects within said        touch screen by applying a first palette;    -   receive a user indication of a swipe gesture via said touch        screen, wherein receiving a user indication further comprises        generating first data representing a first swipe gesture        starting location and second data representing a first swipe        gesture direction, the first swipe gesture starting location        corresponding to a first swipe gesture starting location        coordinate within said first graphical object, wherein said        first graphical object is displayed in a first set of        non-overlapping graphical objects on said touch screen;    -   split said IR image into two parts based on said first data,        wherein the two parts comprises a first part and a second part;    -   select said first part or said second part based on said second        data;    -   image process said selected first part or second part based on        said second data; and;    -   display said image processed first part or displaying said image        processed second part overlaid onto said displayed IR image        within said touch screen.

One of the advantages of the inventions is that complexity and time fora user to find an appropriate imaging processing to apply to an IR imagefor display to a user for analysis may be reduced.

The IR image upon which the analysis is based may be obtained from amemory in the computer system.

In one or more embodiments, the processor is further arranged toretrieve an IR image or a multilayer image from memory 115 as a signalframe of IR image data values or multilayer image data values.

The IR image upon which the analysis is based might be obtained from anexternal thermal imaging device, such as an IR camera, communicativelycoupled to the computer system.

In one or more embodiments, the processor is further arranged to receivean IR image or a multilayer image from a thermal imaging device as asignal frame of IR image data values or multilayer image data values.

The IR image upon which the analysis is based might be obtained from anexternal thermal imaging device, such as an IR camera, communicativelycoupled to the computer system, wherein a video or sequential series ofIR images is received at a predetermined frame rate.

In one or more embodiments, the processor is further arranged to receivea series of IR images or a multilayer images from a thermal imagingdevice as signal frames of IR image data values or multilayer image datavalues with a predefined frame rate.

The IR image upon which the analysis is based might be obtained from anexternal thermal imaging device, such as an IR camera, communicativelycoupled to the computer system, wherein a video or sequential series ofIR images is received at a predetermined frame rate. The series of IRimages may be previously recorded in the thermal imaging device orcaptured live and sent to the computer system.

In one or more embodiments, said received IR image is recorded live inreal-time or near real-time.

In one or more embodiments the processor/processing unit 112 may be aprocessor such as a general or specific purpose processor/processingunit for example a microprocessor, microcontroller or other controllogic that comprises sections of code or code portions, stored on acomputer readable storage medium, such as the memory 115, that are fixedto perform certain tasks but also other alterable sections of code,stored on a computer readable storage medium, that can be altered duringuse. Such alterable sections of code can comprise parameters that are tobe used as input for the various tasks, such as the calibration of thecomputer system, adaption of the sample rate or the filter for thespatial filtering of the images, or any other parameter relatedoperations known to a person skilled in the art and applied withoutinventive skill.

In one or more embodiments, the processor/processing unit 112 isconfigurable using a hardware description language (HDL).

In one or more embodiments, the processor/processing unit 112 is aField-programmable gate array (FPGA), i.e. an integrated circuitdesigned to be configured by the customer or designer aftermanufacturing and configurable using a hardware description language(HDL). For this purpose, embodiments of the invention compriseconfiguration data configured to control an FPGA to perform the stepsand functions of the method embodiments described herein.

In one or more embodiments, said communications interface 116 maycomprise a selection of a Local Area Network (LAN), Metropolitan AreaNetwork (MAN), Global System for Mobile Network (GSM), Enhanced Data GSMEnvironment (EDGE), High Speed Downlink Packet Access (HSDPA), WidebandCode Division Multiple Access (W-CDMA), Code Division Multiple Access(CDMA), Time Division Multiple Access (TDMA), Bluetooth®, Zigbee®,Wi-Fi, Voice over Internet Protocol (VoIP), LTE Advanced, IEEE802.16m,WirelessMAN-Advanced, Evolved High-Speed Packet Access (HSPA+), 3GPPLong Term Evolution (LTE), Mobile WiMAX (IEEE 802.16e), Ultra MobileBroadband (UMB) (formerly Evolution-Data Optimized (EV-DO) Rev. C), FastLow-latency Access with Seamless Handoff Orthogonal Frequency DivisionMultiplexing (Flash-OFDM), High Capacity Spatial Division MultipleAccess (iBurst®) and Mobile Broadband Wireless Access (MBWA) (IEEE802.20) systems, High Performance Radio Metropolitan Area Network(HIPERMAN), Beam-Division Multiple Access (BDMA), World Interoperabilityfor Microwave Access (Wi-MAX) and ultrasonic communication, etc., but isnot limited thereto.

In this document, the terms “computer program product” and“computer-readable storage medium” may be used generally to refer tomedia such as the memory 115 or the storage medium of processing unit112 or an external storage medium. These and other forms ofcomputer-readable storage media may be used to provide instructions toprocessing unit 112 for execution. Such instructions, generally referredto as “computer program code” (which may be grouped in the form ofcomputer programs or other groupings), when executed, enable thecomputer system to perform features or functions of embodiments of thecurrent technology. Further, as used herein, “logic” may includehardware, software, firmware, or a combination of thereof.

In one or more embodiments the memory 115 may comprise a selection of ahard RAM, disk drive, a floppy disk drive, a magnetic tape drive, anoptical disk drive, a CD or DVD drive (R or RW), or other removable orfixed media drive.

The computer system may be configured to capture IR images andalternatively VL images.

FIG. 2 shows a schematic view of one or more embodiments of the computersystem described in FIG. 1, e.g. in the form of a thermographyarrangement, infrared IR camera, smartphone, tablet computer, laptopcomputer, desktop computer or a wrist-worn computer. Said computersystem is configured for processing an infrared (IR) image based on aswipe gesture. The computer system further comprises aprocessor/processing unit 212 provided with specifically designedprogramming or program code portions adapted to control the processingunit 212 to perform the steps and functions of embodiments of the methoddescribed herein. The computer system further comprises at least onememory 215 configured to store data values or parameters received fromthe processor 212 or to retrieve and send data values or parameters tothe processor 212. In one or more embodiments the computer systemfurther comprises a display 217 configured to receive a signal from aprocessor 212 and to display the received signal as a displayed image,e.g. to a user of the computer system. In one or more embodiments thecomputer system further comprises an input device 218 configured toreceive input or indications from a user, e.g. a user to indicate aswipe gesture in an image. In one exemplary embodiment the display 217is integrated with a user input device 218 and implemented as a touchscreen configured to receive a signal from the processor 112 and todisplay the received signal as a displayed image, e.g. to a user of thecomputer system. The touch screen (e.g., display 217) is furtherconfigured to receive indications from a user and to generate dataindicative of user input, thereby enabling the user to interact by userindications with what is displayed, e.g. graphical objects displayed onthe touch screen using various types of gestures. The touch screen(e.g., display 217) is further configured to send the generated data asa signal to said processor 112. In one or more embodiments computersystem further comprises a communications interface 116 configured tosend or receive data values or parameters to/from a processor 112to/from external units via the communications interface 116.

wherein the computer system is further configured to capture infrared(IR) image data values, representing infrared radiation emitted from anobserved real world scene by an infrared (IR) imaging system 213. SaidIR imaging system comprises an infrared (IR) optical system 2131, e.g.comprising a lens, zoom functionality and focus functionality, togetherwith a corresponding infrared IR detector 2132, for example comprising amicro-bolometer focal plane array, arranged to provide an IR image inthe form of a signal frame of IR image data values, representinginfrared radiation emitted from an observed real world scene. Theinfrared (IR) imaging system 213 is further arranged to send the signalframe of IR image data values to a processor 212.

An exemplary embodiment of the operation of such a computer system, suchas an IR camera, is generally as follows: Infrared energy is acceptedvia said infrared optical system 2131 and directed onto the IR detectorelements 2132. Each detector element responds to the infrared radiationor heat energy received. A frame of infrared (IR) image data values may,for example, be captured by scanning all the rows and columns of thedetector and, in some embodiments, analog to digital converted to obtaina captured IR image wherein data values associated to each detectorelement is referred to as an IR image pixel having an associated row andcolumn index.

In one or more embodiments, the computer system further comprises avisible light (VL) imaging system 214 that is configured to capturevisible light (VL) image data values, representing VL emitted from anobserved real world scene. Said VL imaging system employ an visiblelight (VL) optical system 2141, e.g. comprising a lens, zoomfunctionality and focus functionality together with a correspondinginfrared VL detector 2142, for example comprising a digitalcharge-coupled device (CCD) or complementary metal-oxide-semiconductor(CMOS) active pixel sensors, to provide an VL image in the form of asignal frame of VL image data values, representing VL emitted from anobserved real world scene. The VL imaging system 214 is further arrangedto send the signal frame of VL image data values to a processor 212.

In one or more embodiments, the VL imaging system 214 may be adapted tocapture electromagnetic radiation in other non-thermal wavelengths inaddition to or in place of visible light wavelengths. For example, theVL imaging system 214 may be configured to capture near-infrared (NIR)light, short wave infrared (SWIR) light, ultra-violet (UV) light, orother non-thermal light in addition to or in place of visible light. Forsuch embodiments, the VL imaging system 214 may represent a non-thermalimaging system comprising a non-thermal optical system (represented bythe VL optical system 2141) and a non-thermal detector (represented bythe VL detector 2142) responsive to non-thermal radiation. For example,electron multiplying CCD (EMCCD) sensors, scientific CMOS (sCMOS)sensors, intensified charge-coupled device (ICCD) sensors, as well asCCD-based and CMOS-based sensors discussed above and/or other suitablesensors, may be used to implement the non-thermal detector (representedby the VL detector 270) to detect NIR light, SWIR light, and/or othernon-thermal light.

In one or more embodiments, the computer system is configured as twophysically separate devices, i.e. a first device comprising a IR imagingsystem 213 and second device comprising a VL imaging system 214,communicatively coupled and depicting, or capturing, substantially thesame observed real world scene. A memory 215 may be integrated intoeither one or the first or second device or a memory 215 may beintegrated in a physically separate memory device, not shown in thefigure, to which said first and second device is communicativelycoupled.

In one or more embodiments, the computer system is configured to captureinfrared (IR) image data values, representing infrared radiation emittedfrom an observed real world scene and then further to correct orcalibrate captured data values by applying pre-determined IR temperaturecalibration data parameters, to map and scale the captured data valuesfor display as an IR, or thermal, image, singly or combined with a VLimage, according to methods known in the art.

In one or more embodiments, the IR imaging system 213 comprised in thecomputer system (e.g., a thermal imaging device) is further arranged tosend the signal frame of IR image data values to a processor 212intermediate storing in a memory comprised in or separate from thethermal imaging device 210.

In one or more embodiments, the IR imaging system 213 comprised in thecomputer system (e.g., a thermal imaging device) is further arranged tosend the signal frame of IR image data values to an externalprocessor/processing unit (not shown in FIG. 2) from said intermediatestoring via a communications interface 216 (e.g., implemented in a sameor similar manner as communications interface 116).

In one or more embodiments, the processor/processing unit 212 comprisedin the computer system (e.g., a thermal imaging device) is furtherarranged to send the received IR image as a signal frame of IR imagedata values to a an external processor/processing unit (not shown inFIG. 2) directly or from said intermediate storing via saidcommunications interface 216.

In one or more embodiments, a computer system includes a processoroperatively coupled to a memory and a touch screen for processing aninfrared (IR) image based on a swipe gesture, the system comprising:

-   -   a memory;    -   a touch screen;    -   infrared (IR) imaging system; and    -   a processor, wherein said processor is configured to:    -   display an IR image within a first graphical object displayed in        a first set of non-overlapping graphical objects within said        touch screen by applying a first palette;    -   receive a user indication of a swipe gesture via said touch        screen, wherein receiving a user indication further comprises        generating first data representing a first swipe gesture        starting location and second data representing a first swipe        gesture direction, the first swipe gesture starting location        corresponding to a first swipe gesture starting location        coordinate within said first graphical object, wherein said        first graphical object is displayed in a first set of        non-overlapping graphical objects on said touch screen;    -   split said IR image into two parts based on said first data,        wherein the two parts comprises a first part and a second part;    -   select said first part or said second part based on said second        data;    -   image process said selected first part or second part based on        said second data; and;    -   display said image processed first part or displaying said image        processed second part overlaid onto said displayed IR image        within said touch screen.

In one or more embodiments, said computer system further comprises avisible light (VL) imaging system.

In one non-limiting example, an IR imaging system and alternatively a VLimaging system may by integrated in the computer system, therebyenabling capture of IR images and/or VL images.

Method Embodiments

When applying image processing based on swipe gestures, a first part ofthe IR image may be displayed with a first palette or first image modeapplied and a second part of the IR image may be displayed with a secondpalette or second image mode applied, thereby enabling the user tocompare the interpretability of both parts of the IR image.

Image Processing Based on a Swipe Gesture

FIG. 3a illustrates how an IR image is processed based on a swipegesture according to an embodiment of a computer-implemented method in acomputer system having a processor operatively coupled to a memory and atouch screen 300 for processing an infrared (IR) image based on a swipegesture. The method comprises displaying an IR image within a firstgraphical object 310 displayed in a first set of non-overlappinggraphical objects within said touch screen 300 by applying a firstpalette, wherein the method further comprises receiving a userindication of a swipe gesture via said touch screen 300. Said receivingthe user indication further comprises generating first data representinga first swipe gesture starting location 330 and second data representinga first swipe gesture direction 340, the first swipe gesture startinglocation corresponding to a first swipe gesture starting locationcoordinate within said first graphical object 310, wherein said firstgraphical object 310 is displayed in a first set of non-overlappinggraphical objects within said touch screen 300. In one or moreembodiments, the method further comprises displaying menu objects in amenu window, such as icons, slide controls and menus within a secondgraphical object 320. In one non limiting example these menu objects,when indicated by a user, adapts parameters controlling display optionsof the IR image and/or other options of the computer system. Examples ofsuch display options may include setting of color span or selection ofpalette. Examples of other options of the computer system may includesetting image capturing mode to single image or multiple image/video,activation of a spot indicator, activation of an illuminator/light orsetting of image mode, such as IR, VL or combined contrast enhancedimage.

The method further comprises splitting said IR image into two partsbased on said first data, wherein the two parts comprise a first part370 and a second part 360. In one or more embodiments, splitting said IRimage into two parts comprises splitting the IR image along a verticalline 350 intersecting said first swipe gesture starting location 330. Inone or more embodiments, splitting said IR image into two partscomprises splitting the IR image along a horizontal line intersectingsaid first swipe gesture starting location 330 (not shown in thefigure). The method further comprises selecting said first part 370 orsaid second part 360 based on said second data. The method furthercomprises image processing said selected first part 370 or second part360 based on said second data. The method further comprises displayingsaid image processed first part or displaying said image processedsecond part overlaid onto said displayed IR image within said touchscreen. In one or more embodiments, splitting said IR image into twoparts comprises splitting the IR image along a vertical line 350intersecting a horizontally offset location of said first swipe gesturestarting location 330. In one or more embodiments, splitting said IRimage into two parts comprises splitting the IR image along a horizontalline intersecting a vertically offset location of said first swipegesture starting location 330 (not shown in the figure). The first partor the second part may be a single line of the IR image or even null,for example, if said first swipe gesture starting location 330, thevertically offset location, or the horizontally offset locationcoincides with an edge of the IR image.

In one non-limiting example, the horizontally offset location is theright hand side edge of the first graphical object or the left hand sideedge of the first graphical object, such that the entire displayed IRimage is comprised in the first or second part. That is, overlaying animage processed part on the IR image only becomes visible when the swipegesture current location is different from said first swipe gesturestarting location.

A particular image processing operation might be applied when the userperforms a rightward swipe gesture or a leftward swipe gesture, e.g.applying a second palette.

In one or more embodiments, said splitting said IR image into two partscomprises splitting the first parts as a left hand part and splittingthe second part as a right hand part, wherein selecting said first partor said second part comprises determining the direction of the swipegesture as a selection of a rightward swipe gesture or a leftward swipegesture based on said second data and selecting the first part if arightward swipe gesture is determined or selecting the second part if aleftward swipe gesture is determined.

In one or more embodiments, said image processing said selected firstpart or said selected second part based on said second data comprisesapplying a second palette to said selected first part or said selectedsecond part. A particular image processing operation might be appliedwhen the user performs an upward swipe gesture or a downward swipegesture, e.g. applying a second image mode.

In one or more embodiments, said splitting said IR image into two partscomprises splitting the first parts as a top part and splitting thesecond part as a bottom part, wherein selecting said first part or saidsecond part comprises determining the direction of the swipe gesture asa selection of an upward swipe gesture or a downward swipe gesture basedon said second data and selecting the first part if a downward swipegesture is determined or selecting the second part if a upward swipegesture is determined.

In one or more embodiments, said image processing said selected firstpart or said selected second part based on said second data comprisesapplying a second image mode to said selected first part or saidselected second part, wherein said image mode may be a selection of IRonly, fusion, contrast enhancing fusion or VL only but not limitedthereto.

The user may want to visualize only parts of an IR image or to applyimage processing only to parts of an IR image, wherein IR image datavalues of an IR image is within a certain color range.

In one or more embodiments, wherein image processing said selected firstpart or said selected second part based on said second data comprisesapplying a second color range to said selected first part or saidselected second part. This way only particular aspects may be studied,e.g. objects within a particular temperature range.

In some situations, the user might want to analyze various objects inthe displayed IR image by repetitively alternating between differentimage processing of the IR image, e.g. by reversing the direction of theswipe movement repeatedly. By continuously splitting, processing anddisplaying the IR image into a first and a second part as long as theswipe gesture is not ended, the line along which the split is made ismade “sticky” to the user's finger (e.g., to “drag” the vertical linesplitting the IR image) as long as the swipe gesture is not ended.

FIG. 3b illustrates how an IR image is processed based on a swipegesture according to one or more embodiments of a computer-implementedmethod of the computer system described above in connection with FIG. 3a, wherein receiving a user indication of a swipe gesture via said touchscreen further comprises periodically generating third data representinga swipe gesture current location 380, splitting said IR image into afirst part and a second part based on said third data (e.g., along avertical line 390), selecting said first part or said second part basedon said second data, image processing said selected first part or secondpart based on said second data and displaying said image processed firstpart or displaying said image processed second part overlaid onto saiddisplayed IR image within said touch screen 300 until it is detectedthat a fourth data representing a first swipe gesture ending location395 is generated.

In one or more embodiments, the method further comprises displaying saidIR image within a first graphical object displayed in a first set ofnon-overlapping graphical objects within said touch screen by applyingsaid second palette.

One of the advantages of the invention is therefore that the userquickly may alternate between applied palettes or image modes of anobject in the IR image and complete the swipe gesture when a finalselection has been made.

Use Case Embodiments

FIG. 4a shows an example IR image displayed according to one or moreembodiments of a computer-implemented method in a computer system. Asshown, the IR image may be displayed within a first graphical object 410(e.g., an image window or frame) which is displayed in a first set ofnon-overlapping graphical objects (e.g., the image window and a menuwindow 411) within said touch screen by applying a first palette.

In one or more of the embodiments where the IR image is split into afirst part and a second part, the first part may be split as a left handpart and the second part may be split as a right hand part asillustrated for example in FIG. 4 b.

FIG. 4b shows another example IR image illustrating how an IR image maybe split and processed according to one or more embodiments of thecomputer-implemented method in the computer system. The method mayinclude receiving a user indication of a swipe gesture via said touchscreen, wherein said receiving a user indication further comprisesgenerating first data representing a first swipe gesture startinglocation 421 and second data representing a first swipe gesturedirection 422. In one or more embodiments, the first swipe gesturestarting location corresponds to a first swipe gesture starting locationcoordinate within said first graphical object 420, wherein said firstgraphical object is displayed in a first set of non-overlappinggraphical objects on said touch screen. In one or more embodiments, theIR image is split into two parts based on said first data, wherein thetwo parts comprise a first part and a second part, wherein the firstparts are split as a left hand part and the second part is split as aright hand part.

The first part is selected in response to a rightward swipe gesture,wherein the direction of the swipe gesture may be determined as one of arightward swipe gesture or a leftward swipe gesture based on said seconddata.

The selected first part is image processed based on said second data,indicating a rightward swipe, and comprises applying a second palette tosaid selected first part. The image processed first part is displayed orpresented overlaid onto said displayed IR image within the firstgraphical object within said touch screen.

The receiving of the user indication may further comprise periodicallygenerating third data representing a swipe gesture current locationuntil it is detected that a fourth data representing a first swipegesture ending location 423 is generated, in this example when the swipegesture reaches the end of the first graphical object 420 or thetouch-sensitive area of the touch screen. Alternatively, a first swipegesture ending location may be detected when the user ends his swipegesture, e.g. by removing his finger.

For example, when the end of a first graphical object 430 ortouch-sensitive area is detected as the first swipe gesture endinglocation 423, the processed IR image may be displayed with said secondpalette applied, within the first graphical object 430 displayed in afirst set of non-overlapping graphical objects within said touch screen,as shown in FIG. 4 c.

FIG. 5a shows an example IR image displayed according to one or moreembodiments of a computer-implemented method in a computer system. Asshown, the IR image may be displayed within a first graphical object510, such as an image window or frame, displayed in a first set ofnon-overlapping graphical objects, such as the image window and a menuwindow 511, within said touch screen by applying a first palette.

In one or more of the embodiments where the IR image is split into afirst part and a second part, the first part may be split as a top partand the second part as a bottom part as illustrated for example in FIG.5 b.

FIG. 5b shows another example IR image illustrating how an IR image maybe split and processed according to one or more embodiments of thecomputer-implemented method in the computer system. The method mayinclude receiving a user indication of a swipe gesture via said touchscreen, wherein said receiving the user indication further comprisesgenerating first data representing a first swipe gesture startinglocation 521 and second data representing a first swipe gesturedirection 522. In one or more embodiments, the first swipe gesturestarting location 521 corresponds to a first swipe gesture startinglocation coordinate within said first graphical object 520, wherein saidfirst graphical object is displayed in a first set of non-overlappinggraphical objects on said touch screen. In one or more embodiments, theIR image is split into two parts based on said first data, whereinsplitting said IR image into two parts comprises splitting the firstparts as a top part and splitting the second part as a bottom part.

Selecting said first part or said second part further comprisesdetermining the direction of the swipe gesture as a selection of anupward swipe gesture or a downward swipe gesture based on said seconddata, and selecting the first part as a downward swipe gesture isdetermined.

In one or more embodiments, said selected first part is image processedbased on said second data by applying a second image mode to saidselected first part or said selected second part, wherein said imagemode may be a selection of IR only, fusion, contrast enhancing fusion orVL only but not limited thereto. The image processed first part isdisplayed or presented overlaid onto said displayed IR image within thefirst graphical object within said touch screen.

The receiving of the user indication of a swipe via said touch screenmay further comprise periodically generating third data representing aswipe gesture current location until it is detected that a fourth datarepresenting a first swipe gesture ending location 523 is generated, inthis example when the swipe gesture reaches the end of the firstgraphical object 520 or the touch sensitive area of the touch screen.Alternatively, the first swipe gesture ending location may be detectedwhen the user ends his swipe gesture, e.g. by removing his finger.

For example, when the end of a first graphical object 530 ortouch-sensitive area is detected as the first swipe gesture endinglocation 523, the processed IR image may be displayed with said secondimage mode applied, within the first graphical object displayed 530 in afirst set of non-overlapping graphical objects within said touch screen,as shown in FIG. 5 c.

In some embodiments image processing is triggered by a selection of arightward swipe gesture, a leftward swipe, an upward swipe gesture or adownward swipe gesture based on said second data. Further, although inthe examples of FIGS. 4a-5c a rightward swipe triggered image processingto apply a second palette where as a downward swipe triggered imageprocessing to apply an image mode, it should be appreciated that any ofthe rightward, leftward, upward, or downward swipe may be assigned totrigger any desired image processing according to desired applicationsof the embodiments.

FIG. 6 illustrates how an image processing operation may be selectedbased on a determination of a direction of a swipe gesture according toone or more embodiments of the computer-implemented method in thecomputer system described above. In one or more embodiments, saidselecting of said first part or said second part comprises determiningthe direction of the swipe gesture as a selection of a rightward swipegesture, a leftward swipe, an upward swipe gesture or a downward swipegesture based on said second data. In one non-limiting example, anupward or downward swipe gesture 610 would trigger a first type of imageprocessing, e.g. applying a second image mode, and a leftward orrightward swipe gesture 620 would trigger a second type of imageprocessing, e.g. applying a second palette. In this way, a user mayquickly find a palette 630 that enables an improved visualization andinterpretability of an IR image and further quickly find an image mode640 that enables an improved visualization and interpretability of an IRimage.

In some embodiments image processing to resize the IR image is triggeredby stationary swipe gesture or tap gesture.

FIG. 7a shows an example IR image illustrating how an IR image may beprocessed based on a swipe gesture including a stationary swipe or tapgesture, according to one or more embodiments of a computer-implementedmethod in a computer system including a processor operatively coupled toa memory and a touch screen. In one or more embodiments, the method maycomprise:

-   -   displaying an IR image within a first graphical object 710        displayed in a first set of non-overlapping graphical objects        within said touch screen by applying a first palette;    -   receiving a user indication of a swipe gesture via said touch        screen, wherein said receiving the user indication further        comprises generating first data representing a first swipe        gesture starting location and second data representing a first        swipe gesture direction, the first swipe gesture starting        location corresponding to a first swipe gesture starting        location coordinate within said first graphical object, wherein        said first graphical object 710 is displayed in a first set of        non-overlapping graphical objects on said touch screen;    -   splitting said IR image into two parts based on said first data,        wherein the two parts comprise a first part and a second part;    -   selecting said first part or said second part based on said        second data;    -   image processing said selected first part or second part based        on said second data; and    -   displaying said image processed first part or displaying said        image processed second part overlaid onto said displayed IR        image within said touch screen.

In one or more embodiments, said first part comprises the entiredisplayed IR image 750 as shown in FIG. 7b , wherein:

-   -   said selecting said first part or said second part further        comprises determining the direction of the swipe gesture as a        stationary swipe gesture or tap gesture based on said second        data;    -   said image processing said selected first part comprises        resizing said displayed IR image; and;    -   said displaying said image processed first part comprises        displaying said image processed first part within the total area        included by said first graphical object 710 and a second        graphical object 720.

In one or more embodiments, wherein said resizing said displayed IRimage 750 is performed proportionally to a quota based on the areaincluded by said first graphical object 710 and the total area includedby said first graphical object 710 and a second graphical object 720.

The resizing may be performed by image interpolation techniques, aswould be understood by a person skilled in the art.

In one non-limiting example, a user may make a stationary swipe gesturealso referred to as a tap gesture, defined by determining that the firstdata representing a first swipe gesture starting location 730 is equalor substantially equal (e.g., when differences fall within a specifiedsmall margin) to the fourth data representing a first swipe gestureending location 740. The displayed IR image 750 is then resized anddisplayed on an area of the touch screen including a first graphicalobject 710 and a second graphical object 720, e.g. displaying the IRimage in full screen over the previously dedicated to a second graphicalmenu object.

FIG. 8 shows a flowchart of one or more embodiments of acomputer-implemented method in a computer system having a processoroperatively coupled to a memory and a touch screen for processing aninfrared (IR) image based on a swipe gesture. In one or moreembodiments, the method comprises:

At block 810: displaying an IR image within a first graphical objectdisplayed in a first set of non-overlapping graphical objects withinsaid touch screen by applying a first palette;

At block 820: receiving a user indication of a swipe gesture via saidtouch screen, wherein receiving a user indication further comprisesgenerating first data representing a first swipe gesture startinglocation and second data representing a first swipe gesture direction,the first swipe gesture starting location corresponding to a first swipegesture starting location coordinate within said first graphical object,wherein said first graphical object is displayed in a first set ofnon-overlapping graphical objects on said touch screen;

At block 830: splitting said IR image into two parts based on said firstdata, wherein the two parts comprises a first part and a second part;

At block 840: selecting said first part or said second part based onsaid second data;

At block 850: image processing said selected first part or second partbased on said second data; and;

At block 860: displaying said image processed first part or displayingsaid image processed second part overlaid onto said displayed IR imagewithin said touch screen.

Further Embodiments Adjusting Temperature Ranges

In general, processing, presenting, analyzing, and monitoring of an IRimage of an observed real world scene are often dependent on colorranges, temperature spans, temperature ranges, or other parameters thataffect how the IR image may be visually represented. It may therefore bebeneficial to reduce complexity and time for a user to find appropriatecolor ranges, temperature spans, or other image processing/displayingparameters to apply to an IR image, in particular in a computer systemconfigured with a touch screen with a touch-based user inputfunctionality. By conveniently discovering suitable imageprocessing/display parameters for IR images depicting particular scenes,users of a computer system (e.g., an IR camera system or other computersystem configured for thermographic use) may be able to quickly obtain apresentation of the IR images having improved interpretability and/orshowing particular aspects or details that may be analyzed.

FIG. 9 shows various example parameters (e.g., temperature spans) thatmay define IR images or be used for image processing or presenting IRimages in a thermal imaging device display (e.g., display 117 or 217),in accordance with an embodiment of the disclosure. In FIG. 9, variousbars 910-980 may represent different temperature ranges that areinherent in the scene or the IR camera, or levels/spans of differentview parameter settings which can be set by the user of the IR camera.The numerous temperature ranges represented in the example of FIG. 9also serve to illustrate problems associated with presenting IR imageson a display to a user, such as for example a difficulty in interpretingIR images due to complex relationships between the temperature rangeswhich may be hidden from the user.

In FIG. 9, the following temperature ranges are shown, and the notationsintroduced below may be adhered to hereinafter:

Temperature range X (represented by bar 910) illustrates a possiblescene thermal content of the imaged scene. This may substantially be anytemperature range a scene may have.

Temperature range A (represented by bar 920) illustrates an actual scenethermal content of the imaged scene, e.g., a span between the actualminimum temperature in the imaged scene A_(low) and the actual maximumtemperature in the imaged scene A_(high).

Temperature range B (represented by bar 930) illustrates the dynamicrange of the IR camera. B_(low) denotes the lowest detectabletemperature, and B_(high) denotes the highest detectable temperature,i.e., the dynamic range of the imaging system of the IR camera.

Temperature range C (represented by bar 940) illustrates the image rangeof image data values in a captured IR image, e.g. a span between theminimum image data value C_(low) and the maximum image data valueC_(high) in a captured IR image. C_(low) denotes the lowest temperature,and C_(high) denotes the highest temperature in the captured thermalimage content of the IR camera. The actual thermal image content Ccaptured in the IR image may be limited by the temperature range B ofthe thermal imaging device.

Temperature range D (represented by bar 950) illustrates the choice ofwhich temperature span to colorize (e.g., to map temperature values tocolors). This is also referred to herein as a color range or color span,i.e., a mapping to a palette comprising colors. D_(low) denotes thelowest temperature, and D_(high) denotes the highest temperature that isto be colorized by the IR camera.

Temperature range E (represented by bar 960) illustrates the capturedimage's calculated colors to use in the thermal image. E_(low) denotesthe lowest temperature to be colored in the captured image, and E_(high)denotes the highest temperature that is colored in the captured image.However, in the example of FIG. 9, since D_(high) has been set lowerthan C_(high), the part of E located between D_(high) and C_(high) aresaturated, and therefore is colored the same color as the end of thetemperature span D, or a particular color used to display saturatedparts of a thermal image.

Temperature range F (represented by bar 970) illustrates the thermalimage fusion span where thermal image information and visual imageinformation is combined. F_(low) denotes the lower thermal image fusionlevel/threshold, and F_(high) denotes the higher thermal image fusionlevel/threshold. For the infrared image data values within F the thermalimaging device will display IR image data values, and for the infraredimage data values outside F the thermal imaging device will displayvisual light (VL) image data values. Temperature range G (represented bybar 980) illustrates the resulting fusion image presented by the IRcamera.

Temperature ranges A-F may thus respectively comprise a minimumtemperature value, a maximum temperature value, and temperature valuesin between the minimum and the maximum. In general, temperature rangesD-F may be represented by their respective minimum temperature values(also referred to as minimum color levels), maximum temperature values(also referred to as maximum color levels), and optionally one or moretemperature values in between the minimum and the maximum. In onenon-limiting example, temperature ranges D-F may respectively berepresented using a corresponding minimum temperature value, maximumtemperature value, and middle temperature value. According to one ormore embodiments of the disclosure, one or more of temperature rangesA-F may be visually presented (e.g., on a display) to a user, forexample as a separate graphical object such as a legend or scaledisplayed together with an IR image presented according to thetemperature ranges.

As discussed above, it may be beneficial to provide a computer system(e.g., an IR camera or a thermographic arrangement) that permits usersto quickly and conveniently discover and apply desired temperatureranges for processing and/or presenting IR images. Various techniques toprovide such a computer system by processing swipe gestures arediscussed below with reference to FIGS. 10a -14, in accordance withvarious embodiments of the disclosure.

FIG. 10a and FIG. 10b show example images of an IR image and anassociated temperature range that may be displayed within a touchscreen, in accordance with an embodiment of the disclosure. In theexample images of FIGS. 10a and 10b , an IR image and an associatedtemperature range are displayed within graphical objects 1010 and 1016,respectively, on a touch screen. Other graphical objects may bedisplayed if desired. As one example, FIGS. 10a and 10b show a graphicalobject 1020, which may for example be a menu window as shown in FIG. 10b.

Graphical objects 1010, 1016, 1020 may be structured, grouped, managed,and/or arranged for display according to appropriate graphical userinterface (GUI) techniques. For example, graphical objects 1010 and 1020may be managed in a first set of non-overlapping graphical objects,whereas graphical object 1016 may be managed in a second set ofgraphical objects that may overlap with objects in the first set ofgraphical objects.

According to various embodiments, one or more portions of graphicalobject 1016 may each be associated with a corresponding temperaturevalue from the temperature range represented in graphical object 1016.In one example, graphical object 1016 may comprise a first portion 1015associated with a minimum temperature value, a second portion 1017associated with a middle temperature value, and a third portion 1018associated with a maximum temperature value of the temperature rangerepresented in graphical object 1016. In other examples, graphicalobject 1016 may comprise fewer than the three portions identified in theprevious example. That is, for example, there may be just one portionassociated with a temperature value (e.g., a portion associated with amiddle temperature value), or there may just two portions (e.g., twoportions each associated with a maximum and a minimum temperaturevalue). In yet other examples, graphical object 1016 may compriseadditional portions that correspond to additional temperature valuesfrom the temperature range.

According to various embodiments, a user that wishes to adjust one ofthe temperature ranges D-F described in connection with FIG. 9, forexample, may apply a gesture motion on graphical object 1016representing one of the temperature ranges D-F to do so. For example,via an appropriate gesture motion on graphical object 1016, a user maymodify only the minimum value or only the maximum value, orsimultaneously modify any combination of the minimum temperature value,the maximum temperature value, and the middle temperature value of thetemperature range. In one non-limiting example, this may involvestarting a swipe gesture on the first portion 1015, the third portion1018, or the second portion 1017 of graphical object 1016.

FIG. 11a is an example touch screen image illustrating how a swipegesture from a user may be received to modify a temperature range, inaccordance with an embodiment of the present disclosure. The example ofFIG. 11a shows a downward swipe gesture starting on a locationcorresponding to first portion 1015 of graphical object 1016, wherefirst portion 1015 may be associated with a minimum temperature value ofa temperature range represented in graphical object 1016.

FIG. 11b illustrates an example of how the temperature range is modifiedas a result of receiving the user indication of a downward swipe shownin FIG. 11a , in accordance with an embodiment of the disclosure. Invarious embodiments, a first data representing a swipe gesture startinglocation (illustrated in the figures using a cross mark) and a seconddata representing a swipe gesture direction (illustrated in the figuresusing an arrow) may be generated in response to receiving the exampleuser indication of FIG. 11a , where the swipe gesture starting locationcorresponds to first portion 1015 of graphical object 1016 and the swipegesture direction corresponds to a downward direction. Responsive tosuch data representing the example swipe gesture of FIG. 11a , thetemperature range represented in graphical object 1016 may be adjusted,updated, or otherwise modified by lowering its minimum temperaturevalue, according to some embodiments.

As may be understood by comparing FIG. 11a with FIG. 11b , in thisexample the minimum temperature value (indicated numerically in generalproximity to first portion 1015 in this example) is lowered in responseto the downward swipe gesture that started from a location associatedthe minimum temperature value. This in turn lowers a middle temperaturevalue (indicated numerically in general proximity to second portion 1017of graphical object 1016 in this example), while the maximum temperaturevalue (indicated numerically in general proximity to third portion 1018of graphical object 1016 in this example) is unaltered, according toembodiments illustrated by this example. According to variousembodiments, the minimum temperature value may be raised, instead ofbeing lowered, if the swipe gesture direction is an upward directioninstead of the downward direction as given in this example. It should benoted that graphical object 1016 is shown to include a scale withnumerical indications of temperature values (e.g., corresponding tomaximum, middle, and minimum temperature values) only as an example, andthat graphical object 1016 may alternatively or additionally include anyother suitable graphical representation of a desired temperature range,and may or may not include numerical indications, depending onparticular applications of the disclosure.

FIG. 12a illustrates another example of how a swipe gesture from a usermay be received to modify a temperature range, and FIG. 12b illustratesan example of how the temperature range is modified as a result ofreceiving the example swipe gesture of FIG. 12a , in accordance with anembodiment of the present disclosure. The example of FIG. 12a shows adownward swipe gesture starting on a location corresponding to thirdportion 1018 of graphical object 1016, where third portion 1018 may beassociated with a maximum temperature value of a temperature rangerepresented in graphical object 1016.

As illustrated by the examples of FIGS. 12a and 12b , the temperaturerange represented in graphical object 1016 may be adjusted, updated, orotherwise modified by lowering its maximum temperature value (indicatednumerically in general proximity to third portion 1018 of graphicalobject 1016), in response to receiving the downward swipe gesture thatstarted from a location associated with the maximum temperature value,in accordance with some embodiments of the disclosure. This in turnlowers a middle temperature value (indicated numerically in generalproximity to second portion 1017 of graphical object 1016), while theminimum temperature value (indicated numerically in general proximity tofirst portion 1015 of graphical object 1016) is unaltered, according toembodiments illustrated by this example. According to variousembodiments, the maximum temperature value may be raised, instead ofbeing lowered, if the swipe gesture direction is an upward directioninstead of the downward direction as given in this example.

FIG. 13a illustrates yet another example of how a swipe gesture from auser may be received to modify a temperature range, and FIG. 13billustrates an example of how the temperature range is modified as aresult of receiving the example swipe gesture of FIG. 13a , inaccordance with an embodiment of the present disclosure. The example ofFIG. 13a shows a downward swipe gesture starting on a locationcorresponding to second portion 1017 of graphical object 1016, wheresecond portion 1017 may be associated with a middle temperature value ofa temperature range represented in graphical object 1016.

As illustrated by the examples of FIGS. 13a and 13b , the temperaturerange represented in graphical object 1016 may be adjusted, updated, orotherwise modified by lowering its middle temperature value (indicatednumerically in general proximity to second portion 1017 of graphicalobject 1016), in response to receiving the downward swipe gesture thatstarted from a location associated with the middle temperature value, inaccordance with some embodiments of the disclosure. This in turn lowersthe minimum and the maximum temperature values (indicated numerically ingeneral proximity to first portion 1015 and third portion 1018,respectively, of graphical object 1016), according to embodimentsillustrated by this example. According to various embodiments, themiddle, the minimum, and the maximum temperature values may be raised,instead of being lowered, if the swipe gesture direction is an upwarddirection instead of the downward direction as given in this example.

Turning to FIG. 14, a flowchart is illustrated of a computer-implementedmethod 1400 to process an IR image based on a swipe gesture, inaccordance with an embodiment of the disclosure. For example, method1400 may process and/or display IR images based on one or moretemperature ranges that may be adjusted, updated, or otherwise modifiedby processing swipe gestures as illustrated in the examples of FIGS.10a-13b . In various embodiments, method 1400 may comprise thefollowing:

At block 1410, displaying an IR image on a touch screen, based on atemperature range. For example, according to some embodiments, the IRimage may be processed and displayed on the touch screen by applying afirst palette to IR image data values within the temperature range andapplying a second palette to IR image data values outside saidtemperature range. In some embodiments, the IR image may be displayedwithin a first graphical object (e.g., graphical object 1010) displayedon the touch screen. In some embodiments, the first graphical object maybelong to a first set of graphical objects that do not overlap eachother when displayed within the touch screen.

At block 1420, displaying a second graphical object (e.g., graphicalobject 1016) within said touch screen. For example, the second graphicalobject may comprise a scale, legend, or other graphical representationof the temperature range, based on which the IR image was appliedselected palettes. The second graphical object may comprises one or moreportions each associated with a temperature value (e.g., a maximum, amiddle, and a minimum temperature value) from the temperature rangerepresented in the second graphical object. In some embodiments, thesecond graphical object may belong to a second set of graphical objectsthat can be overlapped or overlaid on the first set of graphical objects(e.g., including the IR image) when displayed within the touch screen.

At block 1430, receiving, by a processor communicatively coupled to saidtouch screen, a user indication of a swipe gesture via said touchscreen, wherein said receiving the user indication comprises generatingfirst data representing a swipe gesture starting location and seconddata representing a swipe gesture direction. Block 1430 may furthercomprise determining whether the swipe gesture starting locationrepresented by first data corresponds to a coordinate associated with aselected portion of the second graphical object. For example, said swipegesture starting location may be determined as corresponding to one of afirst portion representing a minimum temperature value of thetemperature range, a second portion representing a middle temperaturevalue of the temperature range, and a third portion representing amaximum temperature value of the temperature range.

At block 1440, adjusting, updating, or otherwise modifying saidtemperature range based on said first data and said second datarepresenting the user indication of the swipe gesture. For example, invarious embodiments, modifying said temperature range may compriselowering or raising one or more of a minimum temperature value, a middletemperature value, and a maximum temperature value of said temperaturerange represented in the second graphical object, based on the swipegesture directions and on which portion of the second graphical objectthe swipe gesture starting location corresponds to, as described abovein connection with the examples of FIGS. 11a -13 b.

At block 1450, image processing said IR image based on said modifiedtemperature range. In the example discussed above in connection withblock 1410, the IR image may be processed by applying the first paletteto IR image data values now within the modified temperature range andapplying the second palette to IR image data values now outside themodified temperature range, according to some embodiments. In otherexamples, one or more additional or alternative imaging processingoperations that are dependent on or associated with the temperaturerange may be performed on the IR image with the now modified temperaturerange.

At block 1460, displaying said image processed IR image within saidtouch screen. Thus, for example, the IR image may be displayed that isupdated or modified by performing imaging processing with the modifiedtemperature range based on the processing of the swipe gesture. In thisway, for example, users may use swipe gestures and view correspondinglyupdated IR images to quickly and conveniently discover and apply desiredtemperature ranges for processing and/or presenting IR images.

In one or more embodiments, the temperature range may be selected fromthe temperature ranges D-F described in relation to FIG. 9. In one ormore embodiments, said second set of graphical objects may beoverlappingly placed or overlaid on the first graphical object at thetop part, bottom part, right part or left part of said first graphicalobject. For example, the second set of graphical objects may include alegend placed at the top part, bottom part, right part or left part ofthe displayed IR image.

In one or more embodiments, said second set of graphical objects mayfurther comprise one or more additional graphical objects. For example,the second set of graphical objects may comprise a graphicalrepresentation of one or more additional temperature ranges (e.g., thetemperature ranges D and F described in relation to FIG. 9 above), and auser may start a swipe gesture starting at a coordinate associated witha selected one of the graphical objects representing the temperatureranges to adjust a desired one of the temperature ranges.

Visualizing Temperature Using a Stationary Swipe Gesture

When image processing, presenting, analyzing, and monitoring an IR imageof an observed real world scene, a spot marker may be overlaid onto orotherwise displayed relative to a displayed IR image to aid a user inmarking or otherwise selecting a particular pixel or group of pixels ofthe displayed IR image. A user may mark or select a particular pixel orgroup of pixels using the spot marker so as to obtain one or moreattributes associated with, or to control processing, presenting, and/oranalyzing of an IR image based on, the marked pixel or group of pixelsof the IR image. For example, the IR image may be processed to read outan IR image data value or temperature associated with the particularpixel or group of pixels marked or selected using the spot marker. Itmay therefore be beneficial to reduce complexity and time for a user toobtain desired attributes of, or control processing, presenting, and/oranalyzing of, an IR image through the use of a spot marker or othersimilar means.

By conveniently obtaining attributes (e.g., temperature values)associated with IR images or conveniently controlling image processingand/or presentation of IR images, users of a computer system (e.g., anIR camera system or other computer system configured for thermographicuse) may be able to quickly analyze particular aspects or details of theIR images and/or to discover a presentation of the IR images havingimproved interpretability. For example, for a user to understand thecaptured observed real worlds scene, it is further advantageous to beable to place a spot marker on a particular pixel or group of pixels ofthe displayed IR image and to be able to read out an IR image data valueor temperature representative of a particular pixel or group of pixels,wherein the IR image data values represents infrared radiation emittedfrom an observed real world scene.

FIG. 15a and FIG. 15b show touch screen images illustrating an exampleof how a user indication of a swipe gesture, such as a stationary swipegesture or a tap gesture, may be received to place or otherwise controla spot marker and present an attribute associated with an IR image, inaccordance with an embodiment of the disclosure. In the example of FIGS.15a and 15b , a user indication may be received of a stationary swipegesture or tap gesture, which may be defined by a swipe gesture startinglocation 1530 (indicated using a cross mark for illustration purposes inthe figures) and a swipe gesture ending location 1550 (indicated using acircle for illustration purposes in the figures) being equal orsubstantially equal (e.g., when differences fall within a specifiedsmall margin) as discussed in connection with FIGS. 7a and 7b above. Inresponse, a graphical object 1511, e.g., a cross hair, hash symbol, orother similar graphical representations, may be displayed to represent aspot marker placed at a location corresponding to the swipe gesturestarting location 1530 or ending location 1550, for example. Graphicalobject 1511 may be overlaid onto graphical object 1010 containing an IRimage.

This example further illustrates that an attribute (e.g., a temperaturevalue, emissivity value, or other IR image attributes) associated withthe IR image at the stationary swipe gesture location (e.g., swipegesture starting location 1530 or ending location 1550) may bedetermined (e.g., based on a single IR image data value or a combinationof IR image data values) and presented in a graphical object 1512,according to some embodiments. The presentation may be numericinformation representing image data values, e.g., in degrees Celsius asshown in FIG. 12b , degrees Fahrenheit, or emissivity. Graphical object1512 may be overlaid onto graphical object 1010 containing the IR image,at a predefined location relative to graphical object 1010, for example.In some embodiments, graphical objects 1010 and 1020 may be managed in afirst set of non-overlapping graphical objects, whereas graphicalobjects 1511 and 1512 may be managed in a second set of graphicalobjects that may overlap with or be overlaid onto objects in the firstset, as discussed above in connection with FIGS. 10a and 10 b.

FIG. 16a and FIG. 16b illustrate another example of how a userindication of a swipe gesture, such as a stationary swipe gesture or atap gesture, may be received to place a spot marker and present anattribute associated with an IR image, in accordance with an embodimentof the disclosure. In this example, in response to receiving astationary swipe gesture or a tap gesture (defined by a swipe gesturestarting location 1630 and ending location 1650 being equal orsubstantially equal), a rectangular area 1613 may be selected based onthe stationary swipe gesture location (e.g., swipe gesture startinglocation 1630 or ending location 1650). In some embodiments, edges ofrectangular area 1613 may be displayed as a graphical object overlaidonto graphical object 1010 containing the IR image, so as to aid theuser in identifying the selected area. Depending on embodiments, thestationary swipe gesture location may determine the center ofrectangular area 1613, a corner of rectangular area 1613, or otherpredefined location relative to rectangular area 1613. Some embodimentsmay allow the user to adjust rectangular area 1613 via a swipe gestureor a dragging gesture starting on a corner or edge of rectangular area1613.

In the example illustrated in FIGS. 16a and 16b , a spot marker locationis determined based on the IR image data values (e.g., pixel values) forpixels of the displayed IR image within rectangular area 1613. In someembodiments, the spot maker location is selected by a predefinedfunction (also referred to herein as a spot marker function) over thepixels within rectangular area 1613. For one or more examples, the spotmarker function may be a minimum image data value function (e.g., afunction that returns a location of a pixel that has the minimum imagedata value over all pixels within rectangular area 1613) or a maximumimage data value function (e.g., a function that returns a location of apixel that has the maximum image data value over all pixels withinrectangular area 1613).

The determined spot marker location may be indicated by a graphicalobject 1611 overlaid onto graphical object 1010 containing the IR image.Graphical object 1611 may be displayed at a location corresponding tothe determined spot marker location, but may otherwise be similar tographical object 1511. Further, according to some embodiments, anattribute (e.g., a temperature value, emissivity value, or other IRimage attributes) associated with the determined spot marker locationmay be presented in graphical object 1512, in a similar manner asdescribed above for FIGS. 15a and 15b . Thus, for example, inembodiments where the spot marker function may be a minimum image datavalue function or a maximum image data value function for IR image data,a cold or a hot spot within rectangular area 1613 may be indicated bygraphical object 1611 and a corresponding temperature value may bepresented as a numerical value in graphical object 1512.

FIG. 17a and FIG. 17b illustrate another example of how a userindication of a swipe gesture, such as a stationary swipe gesture or atap gesture, may be received to place a spot marker (represented by agraphical object 1711) and present an attribute associated with an IRimage, in accordance with an embodiment of the disclosure. The exampleillustrated in FIGS. 17a and 17b shows a circular area 1713 rather thanrectangular area 1613 as the area selected in response to receiving astationary swipe gesture (e.g., defined by a swipe gesture startinglocation 1730 and ending location 1750), but may otherwise be similar tothe example illustrated in FIGS. 16a and 16b . Although rectangular area1613 and circular area 1713 are illustrated as examples of areasselected in response to a stationary swipe gesture for embodimentsdescribed in connection with FIGS. 16a-17b , areas in other shapes arealso contemplated as variations within those embodiments.

FIG. 18 shows an example touch screen image displaying an IR image and aspot marker, in accordance with the various embodiments discussed inconnection with FIGS. 10a-17b above. In this example image, a graphicalobject 1811 in the form of a cross hair or hash symbol is displayed at alocation corresponding to the spot marker location, and a temperaturevalue associated with the spot marker location is displayed in agraphical object 1812 as numerical information in degrees Fahrenheit.

Turning to FIG. 19, a flowchart is illustrated of a computer-implementedmethod 1900 to process an IR image based on a swipe gesture, inaccordance with an embodiment of the disclosure. For example, method1900 may process swipe gestures to place or otherwise control a spotmarker on a displayed IR image and to obtain one or more attributes(e.g., a temperature value) associated with the IR image as illustratedin the examples of FIG. 15a-17b . In some embodiments, method 1900 maycontrol processing, presenting, and/or analyzing of the IR image basedon the spot marker location, in addition to or in place of obtaining anattribute such as a temperature value of the IR image at a locationcorresponding to the spot marker. For example, according to someembodiments, one or more temperature values (e.g., a minimum temperaturevalue, a middle temperature value, and a maximum temperature value) ofthe temperature ranges D-F of FIG. 9 may be set or adjusted based on animage data value of the IR image at the spot marker location, which inturn may affect processing, presenting, and/or analyzing of the IRimage. In various embodiments, method 1900 may comprise the following:

At block 1910, displaying an IR image on a touch screen. For example,according to some embodiments, the IR image may be processed anddisplayed on the touch screen by applying a first palette to IR imagedata values of the IR image. In some embodiments, the IR image may bedisplayed within a first graphical object (e.g., graphical object 1010)displayed on the touch screen. In some embodiments, the first graphicalobject may belong to a first set of graphical objects that do notoverlap each other when displayed within the touch screen.

At block 1920, receiving, by a processor communicatively coupled to saidtouch screen, a user indication of a swipe gesture via said touchscreen, wherein said receiving the user indication comprises generatingfirst data representing a swipe gesture starting location and seconddata representing a swipe gesture direction. In some embodiments, block1920 may also comprise determining whether the received user indicationcorresponds to a stationary swipe gesture or a tap gesture. Depending onembodiments, such determination may be made when the swipe gesturestarting location equals or substantially equals (e.g., when differencesfall within a small range) a swipe gesture ending location and/or basedon the swipe gesture direction (e.g., when no or negligible swipegesture direction is detected). Block 1920 may further involveassociating the swipe gesture starting location with a coordinate withinthe first graphical object (e.g., within graphical object 1010containing the IR image).

At block 1930, determining a spot marker location, in response toreceiving a user indication corresponding to a stationary swipe or tapgesture. In some embodiments, the spot marker location may be determinedrelative to the swipe gesture starting location or the swipe gestureending location of the stationary swipe gesture. For example, the spotmarker location may be set as the swipe gesture starting location (orthe ending location, which may be the same or substantially the same asthe starting location for a stationary swipe gesture). In someembodiments, the stationary swipe gesture may define a selected area(e.g., rectangular area 1613 or circular area 1713) rather than the spotmarker location, as discussed above in connection with FIGS. 16a-17b .In such embodiments, the spot marker location may then be determinedbased on processing of IR image data values (e.g., pixel values) forpixels of the IR image within the selected area defined by thestationary swipe gesture, as discussed above in connection with FIGS.16a-17b . Such processing of IR image data values may involve applying aspot marker function such as a minimum image data value function, amaximum image data value function, or other functions over aggregate IRimage data values for pixels within the selected area, as discussedherein above.

At block 1940, displaying a second graphical object (e.g., graphicalobject 1511, 1611, 1711, or 1811) including a cross hair, a hash symbol,or other graphical representation of a spot marker at the spot markerlocation. For example, the second graphical object representing a spotmarker may be overlaid onto the first graphical object displaying the IRimage, at a location corresponding to the spot marker location in thedisplayed IR image. In some embodiments, the second graphical object maybelong to a second set of graphical objects that can be overlapped oroverlaid on the first set of graphical objects (e.g., the displayed IRimage) when displayed within the touch screen. In some embodiments,block 1940 may also involve overlaying a third graphical objectrepresenting the contour or edges of the selected area (e.g.,rectangular area 1613 or circular area 1713) onto the first graphicalobject containing the IR image.

At block 1950, processing the IR image based on the spot markerlocation. In one or more embodiments, this may include obtaining anattribute (e.g., a temperature value) associated with a pixel or groupof pixels of the IR image, where the pixel or group of pixels isselected based on the spot marker location, as discussed above inconnection with the examples of FIGS. 15a -18 above. In someembodiments, the obtained attribute may be presented in a fourthgraphical object (e.g., graphical object 1512), which may for example beoverlaid on the first graphical object containing the IR image. In oneexample, the fourth graphical object may contain a numericalrepresentation of the obtained attribute, such as the temperature valuesin degrees Celsius or Fahrenheit.

Expanding or Minimizing a Legend Using a Swipe Gesture.

When image processing, presenting, analyzing, and monitoring an IR imageof an observed real world scene, the requirement of the relative sizesof graphical objects, e.g., between the displayed image and the legend,may be adjusted as necessary. For example, in some handheld infraredimaging devices, the display area may be limited, and it may thus beadvantageous to present some graphical objects such as a legend or amenu selectively in an expanded or minimized form.

FIG. 20a and FIG. 20b show touch screen images illustrating an exampleof how a size of a graphical object may be adjusted based on a swipegesture, in accordance with an embodiment of the disclosure. FIG. 20ashows a graphical object in a reduced or minimized form 1015 a, whereasFIG. 20b shows the same graphical object in an expanded or full form1015 b. In this example, the graphical object representing a legend maybe switched from its reduced or minimized form 1015 a to its expanded orfull form 1015 b in response to receiving a swipe gesture having a swipegesture starting location 2030, a swipe gesture direction 2040, and aswipe gesture ending location 2040 as shown in FIG. 20 a.

In particular, the swipe gesture starting location 2030 may correspondto a coordinate associated with the graphical object in its reduced orminimized form 1015 a, with the swipe gesture direction 2040corresponding to an approximate direction in which the graphical objectis to be expanded and the swipe gesture ending location 2060corresponding to a coordinate outside a predefined threshold (e.g., toindicate a confirmation of a switch to an expanded form).

Conversely, by receiving and processing a user indication of a swipegesture starting on the graphical object in its expanded or full form1015 b, with a swipe direction corresponding to an approximate directionin which the graphical object is to be reduced or minimized and a swipeending location being past a predefined threshold, the graphical objectin its expanded or full form 1015 b may be switched to its reduced orminimized form 1015 a.

FIG. 21 illustrates a flowchart of a computer-implemented method 2100 toprocess graphical objects displayed on a touch screen based on a swipegesture, in accordance with an embodiment of the disclosure. Forexample, method 2100 may process graphical objects displayed, includingan IR image and a graphical representation of a legend or menu, toadjust the relative sizes of the graphical objects in response toreceiving a swipe gesture. In various embodiments, method 2100 maycomprise the following:

At block 2110, displaying an IR image on a touch screen. For example,the IR image may be processed and displayed within a first graphicalobject (e.g., graphical object 1010) displayed on the touch screen. Insome embodiments, the first graphical object may belong to a first setof graphical objects that do not overlap each other when displayedwithin the touch screen. In some embodiments, the IR image may beprocessed and displayed by applying a first palette to IR image datavalues within a first temperature range and applying a second palette toIR image data values outside said first temperature range.

At block 2120, displaying a second graphical object within said touchscreen, where the second graphical object may be displayed either in itsfirst form (e.g., reduced or minimized form 1015 a) or in its secondform (e.g., expanded or full form 1015 b). For example, the secondgraphical object may comprise a scale, legend, or other graphicalrepresentation of the first temperature range, based on which the IRimage was applied selected palettes. In some embodiments, the secondgraphical object may belong to a second set of graphical objects thatcan be overlapped or overlaid on the first set of graphical objects(e.g., the displayed IR image) when displayed within the touch screen.

At block 2130, receiving, by a processor communicatively coupled to saidtouch screen, a user indication of a swipe gesture via said touchscreen, wherein said receiving the user indication comprises generatingfirst data representing a swipe gesture starting location and seconddata representing a swipe gesture direction. In some embodiments, block2130 may further comprise determining whether the first swipe gesturestarting location represented by the first data corresponds to acoordinate associated with the second graphical object and whether theswipe gesture direction represented by the second data corresponds toone of predefined directions associated with an indication to adjust thesize of the second graphical object.

At block 2140, modifying said second graphical object based on saidfirst data and said second data, wherein modifying said second graphicalobject comprises modifying the size of the second graphical object toits first form (e.g., reduced or minimized form 1015 a) or to its secondform (e.g., expanded or full form 1015 b). For example, the size of thesecond graphical object may be changed from its predefined reduced orminimized size to its predefined expanded or full size, or vice-versa,based upon a determination that the first data representing the swipegesture starting location corresponds to a coordinate associated withthe second graphical object and that the second data representing theswipe gesture direction corresponds to one of predefined directionsassociated with an indication to adjust the size of the second graphicalobject.

Adjusting Multiple Parameter Ranges Using Spatial Separation on theTouch Screen

As discussed, processing, presenting, analyzing, and monitoring of an IRimage of an observed real world scene are often dependent on multipleparameters that control the image processing of a visual representationof the IR image. It may therefore be beneficial to reduce complexity andtime for a user to adjust a plurality of parameters that control avisual representation of an IR image, in particular in a computer systemconfigured with a touch screen with a touch-based user inputfunctionality. By conveniently viewing and controlling multipleparameters that affect how IR images are processed and presented, usersof a computer system (e.g., an IR camera system or other computer systemconfigured for thermographic use) may be able to quickly obtain apresentation of the IR images having improved interpretability and/orshowing particular aspects or details that may be analyzed.

FIG. 22a and FIG. 22b show touch screen images illustrating an exampleof how a swipe gesture from a user may be received to adjust one of aplurality of parameters that control or affect processing and/orpresentation of an IR image, in accordance with an embodiment of thedisclosure. In the example of FIGS. 22a and 22b , an IR image may bedisplayed within graphical object 1010 as discussed above. Additionalgraphical objects, such as graphical object 1020 representing forexample a menu, may also be displayed if desired. As discussed above,graphical objects 1010 and 1020 may be managed in a first set ofnon-overlapping graphical objects according to one or more embodiments.

In this example, two parameters are each graphically represented ingraphical objects 22151 and 22152, respectively. Additional parametersmay be displayed in additional graphical objects if desired, withoutdeparting from the scope and spirit of the present disclosure. Theparameters in this example each define a range of parameter values, andmay be presented as a scale or legend (as shown in FIGS. 22a and 22b )or other graphical representation that corresponds to the range ofparameter values. According to some embodiments, graphical objects 22151and 22152 may overlap or be overlaid onto graphical object 1010displaying the IR image. In this regard, graphical objects 22151 and22152 may be managed in a second set of graphical objects that mayoverlap with objects in the first set of graphical objects, according toone or more embodiments.

According to various embodiments, one or more portions of graphicalobject 22151 may each be associated with a parameter value from therange of parameter values represented in graphical object 22151. In oneexample, graphical object 22151 may comprise a first portion 22161associated with a minimum parameter value, a second portion 22171associated with a middle parameter value, and a third portion 22181associated with a maximum parameter value of the parameter rangerepresented in graphical object 22151. Similarly, graphical object 22152may comprise a first portion 22162 associated with a minimum parametervalue, a second portion 22172 associated with a middle parameter value,and a third portion 22182 associated with a maximum parameter value ofthe parameter range represented in graphical object 221521. In otherexamples, graphical objects 22151 and/or 22152 may comprise fewer thanthe three portions identified in the previous example. That is, forexample, there may be just one portion associated with a parameter value(e.g., a portion associated with a middle parameter value), or there mayjust two portions (e.g., two portions each associated with a maximum anda minimum parameter value). In yet other examples, graphical object 1016may comprise additional portions that correspond to additional parametervalues from the parameter range.

According to various embodiments, a user that wishes to adjust one ofthe parameter ranges represented in graphical objects 22151 and 22152may apply a gesture motion on an appropriate one of graphical objects22151 and 22152 to do so. For example, via an appropriate gesture motionon graphical object 22151/22152, a user may modify only the minimumvalue or only the maximum value, or simultaneously modify anycombination of the minimum value, the maximum value, and the middlevalue of the parameter range. As a non-limiting example, this mayinvolve starting a swipe gesture on the first portion 22161/22162, thethird portion 22181/22182, or the second portion 22171/22172 ofgraphical object 22181/22182. The example of FIGS. 22a and 22b showsthat a downward swipe gesture starting on the first portion 22161 ofgraphical object 22151 may be received and processed to lower theminimum parameter value from 0.3. to 0, which in turn may lower themiddle parameter value of the parameter range. According to variousembodiments, the minimum temperature value may be raised, instead ofbeing lowered, if the swipe gesture direction is an upward directioninstead of the downward direction as given in this example.

It should be noted that graphical objects 22151 and/or 22152 are shownto include a scale with numerical indications of parameter values (e.g.,corresponding to maximum, middle, and minimum parameter values) only asan example, and that graphical objects 22151 and/or 22152 mayalternatively or additionally include any other suitable graphicalrepresentation of a desired parameter range, and may or may not includenumerical indications, depending on particular applications of thedisclosure. Further, although the parameters in the example of FIGS. 22aand 22b are shown to comprise a range of parameter values, thetechniques disclosed herein may also be applied to parameters thatcomprise discrete values (e.g., including binary values such as on/offor yes/no). For example, a swipe gesture starting on a particularportion of a graphical object representing a parameter may be processedto adjust one or more discrete values of the parameter.

FIGS. 23a and 23b illustrate another example of how a swipe gesture froma user may be received to modify another parameter, in accordance withan embodiment of the present disclosure. The example of FIGS. 23a and23b shows a downward swipe gesture on graphical object 22152 instead ofgraphical object 22151. Thus, in this example, the downward swipegesture starting on the first portion 22162 of graphical object 22152 isreceived and processed to adjust the minimum value of the parameterrange represented in graphical object 22152 from 30% to 0%, which inturn adjusts the middle value of the parameter range.

FIG. 24 shows a flowchart of a computer-implemented method 2400 toprocess an IR image based on a swipe gesture, in accordance with anembodiment of the disclosure. For example, method 2400 may processand/or display IR images based on a plurality of parameter ranges thatmay be adjusted, updated, or otherwise modified by processing swipegestures as illustrated in the examples of FIGS. 22a-23b . In variousembodiments, method 2400 may comprise the following:

At block 2410, displaying an IR image on a touch screen based on a firstparameter range and a second parameter range. For example, according tosome embodiments the IR image may be processed based on the first andthe second parameter ranges, and displayed within a first graphicalobject (e.g., graphical object 1010) displayed on the touch screen. Insome embodiments, the first graphical object may belong to a first setof graphical objects that do not overlap each other when displayedwithin the touch screen.

At block 2420, displaying a second graphical object (e.g., graphicalobject 22151) and a third graphical object (e.g., graphical object22152) within said touch screen. For example, the second and the thirdgraphical objects may each comprise a scale, legend, or other graphicalrepresentation of the first and the second parameter ranges,respectively. In some embodiments, the second and the third graphicalobjects may belong to a second set of graphical objects that can beoverlapped or overlaid on the first set of graphical objects (e.g., thedisplayed IR image) when displayed within the touch screen,

At block 2430, receiving, by a processor communicatively coupled to saidtouch screen, a user indication of a swipe gesture via said touchscreen, wherein said receiving the user indication comprises generatingfirst data representing a swipe gesture starting location and seconddata representing a swipe gesture direction. Block 2430 may furthercomprise determining whether the swipe gesture starting locationrepresented by first data corresponds to a coordinate associated with aselected portion of the second or the third graphical object. Forexample, said swipe gesture starting location may be determined ascorresponding to a first portion, a second portion, or a third portionof the second or the third graphical object, where each portion isassociated with a minimum parameter value, a middle parameter value, ora maximum parameter value of the first or the second parameter range.

At block 2440, adjusting, updating, or otherwise modifying said first orsaid second parameter range based on said first data and said seconddata representing the user indication of the swipe gesture. For example,in various embodiments, modifying said first or said second parameterrange may comprise lowering or raising one or more of a minimumparameter value, a middle parameter value, and a maximum parameter valueof said first or said second parameter range, based on the swipedirection and on which portion of the second or the third graphicalobject the swipe gesture starting location corresponds to, asillustrated in the examples of FIGS. 22a-23b and the corresponding textabove.

At block 2450, image processing said IR image based on said the first orthe second parameter range as modified, and at block 2460, displayingsaid image processed IR image within said touch screen. Thus, forexample, the IR image may be displayed that is updated or modified byperforming imaging processing with the first and/or the second parameterranges that may have been modified according to the swipe gesturereceived from the user via the touch screen. In this way, for example,users may use swipe gestures and view correspondingly updated IR imagesto quickly and conveniently discover and apply desired parameter rangesfor processing and/or presenting IR images.

Gesture Manipulation of an Infrared Imaging Device

In one or more embodiments, a gesture on a touchscreen may be receivedand processed to control infrared imaging device functionality. In onenon-limiting example, any of the gestures including a tap, double tap,drag, flick, pinch, spread, press, press+tap, press+drag, or rotate maybe received to control a function in an infrared imaging device, such ascontrast adjustment, fusion, blending, contrast enhanced fusion,panorama stitching, global positioning system (GPS) functionality, laserillumination functionality, video control functionality, communicationfunctionality, annotation functionality, focus and zoom functionality,insulation or moisture analysis functionality, or other functionalitythat may be provided on particular infrared imaging devices.

Aligning

Since the capturing of the infrared (IR) image and capturing of thevisual light (VL) image is generally performed by different imagingsystems of the thermal imaging device mounted in a way that the offset,direction and rotation around the optical axes differ. The optical axesbetween the imaging systems may be at a distance from each other and anoptical phenomenon known as parallax distance error will arise. Theoptical axes between the imaging systems may be oriented at an angle inrelation to each other and an optical phenomenon known as parallaxpointing error will arise. The rotation of the imaging systems aroundtheir corresponding optical axes and an optical phenomenon known asparallax rotation error will arise. Due to these parallax errors thecaptured view of the real world scene, called field of view (FOV) mightdiffer between the IR imaging system and the VL imaging system.

Since the capturing of the infrared (IR) image and capturing of thevisible light (VL) image is generally performed by different imagingsystems of the thermal imaging device with different optical systemswith different properties, such as magnification, the captured view ofthe real world scene, called field of view (FOV) might differ betweenthe imaging systems. The IR image and the VL image might be obtainedwith different optical systems with different optical properties, suchas magnification, resulting in different sizes of the FOV captured bythe IR sensor and the VL sensor.

In order to combine the captured IR and captured VL image the imagesmust be adapted so that an adapted IR image and adapted VL imagerepresenting the same part of the observed real world scene is obtained,in other words compensating for the different parallax errors and FOVsize. This processing step is referred to as registration of oralignment of the IR image and the VL image.

Registration or alignment can be performed according to any method knownto a skilled person in the art.

Where applicable, various embodiments provided by the present disclosurecan be implemented using hardware, software, or combinations of hardwareand software. Also where applicable, the various hardware componentsand/or software components set forth herein can be combined intocomposite components comprising software, hardware, and/or both withoutdeparting from the spirit of the present disclosure. Where applicable,the various hardware components and/or software components set forthherein can be separated into sub-components comprising software,hardware, or both without departing from the spirit of the presentdisclosure. In addition, where applicable, it is contemplated thatsoftware components can be implemented as hardware components, andvice-versa.

Computer Readable Media

Software in accordance with the present disclosure, such asnon-transitory instructions, program code, and/or data, can be stored onone or more non-transitory machine readable mediums. It is alsocontemplated that software identified herein can be implemented usingone or more general purpose or specific purpose computers and/orcomputer systems, networked and/or otherwise. Where applicable, theordering of various steps described herein can be changed, combined intocomposite steps, and/or separated into sub-steps to provide featuresdescribed herein.

In one or more embodiments, a computer program product for processing aninfrared (IR) image based on a swipe gesture comprising computerreadable code configured to, when executed in a processor, perform anyor all of the method steps described herein.

In one or more embodiments, a non-transitory computer readable memory orprocessing an infrared (IR) image based on a swipe gesture on which isstored computer readable code configured to, when executed in aprocessor, perform any or all of the method steps described herein.

Where applicable, various embodiments provided by the present disclosurecan be implemented using hardware, software, or combinations of hardwareand software. Also where applicable, the various hardware componentsand/or software components set forth herein can be combined intocomposite components comprising software, hardware, and/or both withoutdeparting from the spirit of the present disclosure. Where applicable,the various hardware components and/or software components set forthherein can be separated into sub-components comprising software,hardware, or both without departing from the spirit of the presentdisclosure. In addition, where applicable, it is contemplated thatsoftware components can be implemented as hardware components, andvice-versa.

Software in accordance with the present disclosure, such asnon-transitory instructions, program code, and/or data, can be stored onone or more non-transitory machine readable mediums. It is alsocontemplated that software identified herein can be implemented usingone or more general purpose or specific purpose computers and/orcomputer systems, networked and/or otherwise. Where applicable, theordering of various steps described herein can be changed, combined intocomposite steps, and/or separated into sub-steps to provide featuresdescribed herein.

Embodiments described above illustrate but do not limit the invention.It should also be understood that numerous modifications and variationsare possible in accordance with the principles of the invention.Accordingly, the scope of the invention is defined only by the followingclaims.

What is claimed is:
 1. A method of processing an infrared (IR) imagebased on a swipe gesture, the method comprising: displaying an IR imageon a touch screen, wherein the IR image is displayed within a firstgraphical object of one or more graphical objects displayed within saidtouch screen; and receiving, by a processor communicatively coupled tosaid touch screen, a user indication of a swipe gesture via said touchscreen, wherein said receiving the user indication comprises generatingfirst data representing a first swipe gesture starting location andsecond data representing a first swipe gesture direction.